Estimating Causal Installed-Base Effects: A Bias-Correction Approach

New empirical models of consumer demand that incorporate social preferences, observational learning, word-of-mouth or network effects have the feature that the adoption of others in the reference group - the “installed-base” - has a causal effect on current adoption behavior. Estimation of such causal installed-base effects is challenging due to the potential for spurious correlation between the adoption of agents, arising from endogenous assortive matching into social groups (or homophily) and from the existence of unobservables across agents that are correlated. In the absence of experimental variation, the preferred solution is to control for these using a rich specification of fixed-effects, which is feasible with panel data. We show that fixedeffects estimators of this sort are inconsistent in the presence of installed-base effects; in our simulations, random-effects specifications perform even worse. Our analysis reveals the tension faced by the applied empiricist in this area: a rich control for unobservables increases the credibility of the reported causal effects, but the incorporation of these controls introduces biases of a new kind in this class of models. We present two solutions: an instrumental variable approach, and a new bias-correction approach, both of which deliver consistent estimates of causal installed-base effects. The bias-correction approach is tractable in this context because we are able to exploit the structure of the problem to solve analytically for the asymptotic bias of the installed-base estimator, and to incorporate it into the estimation routine. Our approach has implications for the measurement of social effects using non-experimental data, and for measuring marketing-mix effects in the presence of state-dependence in demand, more generally. Our empirical application to the adoption of the Toyota Prius Hybrid in California reveals evidence for social influence in diffusion, and demonstrates the importance of incorporating proper controls for the biases we identify

Advertising Content and Consumer Engagement on Social Media: Evidence from Facebook

We describe the effect of social media advertising content on customer engagement using data from Facebook. We content-code 106,316 Facebook messages across 782 companies, using a combination of Amazon Mechanical Turk and natural language processing algorithms. We use this data set to study the association of various kinds of social media marketing content with user engagement—defined as Likes, comments, shares, and click-throughs—with the messages. We find that inclusion of widely used content related to brand personality—like humor and emotion—is associated with higher levels of consumer engagement (Likes, comments, shares) with a message. We find that directly informative content—like mentions of price and deals—is associated with lower levels of engagement when included in messages in isolation, but higher engagement levels when provided in combination with brand personality–related attributes. Also, certain directly informative content, such as deals and promotions, drive consumers’ path to conversion (click-throughs). These results persist after incorporating corrections for the nonrandom targeting of Facebook’s EdgeRank (News Feed) algorithm and so reflect more closely user reaction to content than Facebook’s behavioral targeting. Our results suggest that there are benefits to content engineering that combines informative characteristics that help in obtaining immediate leads (via improved click-throughs) with brand personality–related content that helps in maintaining future reach and branding on the social media site (via improved engagement). These results inform content design strategies. Separately, the methodology we apply to content-code text is useful for future studies utilizing unstructured data such as advertising content or product reviews

Online Evaluation of Audiences for Targeted Advertising via Bandit Experiments

Firms implementing digital advertising campaigns face a complex problem in determining the right match between their advertising creatives and target audiences. Typical solutions to the problem have leveraged non-experimental methods, or used "split-testing" strategies that have not explicitly addressed the complexities induced by targeted audiences that can potentially overlap with one another. This paper presents an adaptive algorithm that addresses the problem via online experimentation. The algorithm is set up as a contextual bandit and addresses the overlap issue by partitioning the target audiences into disjoint, non-overlapping sub-populations. It learns an optimal creative display policy in the disjoint space, while assessing in parallel which creative has the best match in the space of possibly overlapping target audiences. Experiments show that the proposed method is more efficient compared to naive "split-testing" or non-adaptive "A/B/n" testing based methods. We also describe a testing product we built that uses the algorithm. The product is currently deployed on the advertising platform of JD.com, an eCommerce company and a publisher of digital ads in China

Advertising Media and Target Audience Optimization via High-dimensional Bandits

We present a data-driven algorithm that advertisers can use to automate their digital ad-campaigns at online publishers. The algorithm enables the advertiser to search across available target audiences and ad-media to find the best possible combination for its campaign via online experimentation. The problem of finding the best audience-ad combination is complicated by a number of distinctive challenges, including (a) a need for active exploration to resolve prior uncertainty and to speed the search for profitable combinations, (b) many combinations to choose from, giving rise to high-dimensional search formulations, and (c) very low success probabilities, typically just a fraction of one percent. Our algorithm (designated LRDL, an acronym for Logistic Regression with Debiased Lasso) addresses these challenges by combining four elements: a multiarmed bandit framework for active exploration; a Lasso penalty function to handle high dimensionality; an inbuilt debiasing kernel that handles the regularization bias induced by the Lasso; and a semi-parametric regression model for outcomes that promotes cross-learning across arms. The algorithm is implemented as a Thompson Sampler, and to the best of our knowledge, it is the first that can practically address all of the challenges above. Simulations with real and synthetic data show the method is effective and document its superior performance against several benchmarks from the recent high-dimensional bandit literature.Comment: 39 pages, 8 figure

Estimating Causal Installed-Base Effects: A Bias-Correction Approach

New empirical models of consumer demand that incorporate social preferences, observational learning, word-of-mouth or network effects have the feature that the adoption of others in the reference group - the “installed-base” - has a causal effect on current adoption behavior. Estimation of such causal installed-base effects is challenging due to the potential for spurious correlation between the adoption of agents, arising from endogenous assortive matching into social groups (or homophily) and from the existence of unobservables across agents that are correlated. In the absence of experimental variation, the preferred solution is to control for these using a rich specification of fixed-effects, which is feasible with panel data. We show that fixedeffects estimators of this sort are inconsistent in the presence of installed-base effects; in our simulations, random-effects specifications perform even worse. Our analysis reveals the tension faced by the applied empiricist in this area: a rich control for unobservables increases the credibility of the reported causal effects, but the incorporation of these controls introduces biases of a new kind in this class of models. We present two solutions: an instrumental variable approach, and a new bias-correction approach, both of which deliver consistent estimates of causal installed-base effects. The bias-correction approach is tractable in this context because we are able to exploit the structure of the problem to solve analytically for the asymptotic bias of the installed-base estimator, and to incorporate it into the estimation routine. Our approach has implications for the measurement of social effects using non-experimental data, and for measuring marketing-mix effects in the presence of state-dependence in demand, more generally. Our empirical application to the adoption of the Toyota Prius Hybrid in California reveals evidence for social influence in diffusion, and demonstrates the importance of incorporating proper controls for the biases we identify

Online Causal Inference for Advertising in Real-Time Bidding Auctions

Real-time bidding (RTB) systems, which leverage auctions to programmatically allocate user impressions to multiple competing advertisers, continue to enjoy widespread success in digital advertising. Assessing the effectiveness of such advertising remains a lingering challenge in research and practice. This paper presents a new experimental design to perform causal inference on advertising bought through such mechanisms. Our method leverages the economic structure of first- and second-price auctions, which are ubiquitous in RTB systems, embedded within a multi-armed bandit (MAB) setup for online adaptive experimentation. We implement it via a modified Thompson sampling (TS) algorithm that estimates causal effects of advertising while minimizing the costs of experimentation to the advertiser by simultaneously learning the optimal bidding policy that maximizes her expected payoffs from auction participation. Simulations show that not only the proposed method successfully accomplishes the advertiser's goals, but also does so at a much lower cost than more conventional experimentation policies aimed at performing causal inference

Parallel Experimentation in a Competitive Advertising Marketplace

When multiple firms are simultaneously running experiments on a platform, the treatment effects for one firm may depend on the experimentation policies of others. This paper presents a set of causal estimands that are relevant to such an environment. We also present an experimental design that is suitable for facilitating experimentation across multiple competitors in such an environment. Together, these can be used by a platform to run experiments "as a service," on behalf of its participating firms. We show that the causal estimands we develop are identified nonparametrically by the variation induced by the design, and present two scalable estimators that help measure them in typical high-dimensional situations. We implement the design on the advertising platform of JD.com, an eCommerce company, which is also a publisher of digital ads in China. We discuss how the design is engineered within the platform's auction-driven ad-allocation system, which is typical of modern, digital advertising marketplaces. Finally, we present results from a parallel experiment involving 16 advertisers and millions of JD.com users. These results showcase the importance of accommodating a role for interactions across experimenters and demonstrates the viability of the framework

Social Ties and User Generated Content: Evidence from an Online Social Network

We use variation in wind speeds at surfing locations in Switzerland as exogenous shifters of users' propensity to post content about their surfing activity onto an online social network. We exploit this variation to test whether users' social ties on the network have a causal effect on their content generation, and whether conent generation in turn has a similar causal effect on the users' abilty to form social ties. Economically significant causal effects of this kind can produce positive feedback that generate multiplier e¤ects to interventions that subsidize tie formation. We argue these interventions can therefore be the basis of a strategy by the rm to indirectly faciliate content generation on the site. The exogenous variation provided by wind speeds enable us to measure this feedback empirically and to assess the return on investment from such policies. We use a detailed dataset from an online social network that comprises the complete details of social tie formation and content generation on the site. The richness of he data enable us to control for several spurious confounds that have typically plagued empirical analysis of social interactions. Our results show evidence for significant positive feedback in user generated content. We discuss the implications of the estimates for the management of the content and the growth of the network

Comparison Lift: Bandit-based Experimentation System for Online Advertising

Comparison Lift is an experimentation-as-a-service (EaaS) application for testing online advertising audiences and creatives at JD.com. Unlike many other EaaS tools that focus primarily on fixed sample A/B testing, Comparison Lift deploys a custom bandit-based experimentation algorithm. The advantages of the bandit-based approach are two-fold. First, it aligns the randomization induced in the test with the advertiser's goals from testing. Second, by adapting experimental design to information acquired during the test, it reduces substantially the cost of experimentation to the advertiser. Since launch in May 2019, Comparison Lift has been utilized in over 1,500 experiments. We estimate that utilization of the product has helped increase click-through rates of participating advertising campaigns by 46% on average. We estimate that the adaptive design in the product has generated 27% more clicks on average during testing compared to a fixed sample A/B design. Both suggest significant value generation and cost savings to advertisers from the product